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(54) Method and apparatus for segmenting data to create mixed raster content planes 



(57) An improved technique for compressing a color two portions at the edges. One plane contains image 

or gray scale pixel map representing a document using data for the dark sides of the edges, while image data 

an MRC format includes a method of segmenting an for the bright sides of the edges and the smooth portions 

original pixel map into two planes (12,16), and then com- of the image are placed on the other plane. This results 

pressing the data or each plane in an efficient manner. in improved image compression ratios and enhanced 

The image is segmented by separating the image into image quality. 
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Description 

[0001]- This - invention relates generally to image 
processing and, more particularly, 1 to techniques lor seg- 
menting, classifying arid/or compressing trie digital rep- 
resentation of a document. ■ ' 

[0002] Documents scanned at high resolutions re : 
quire very large' amounts of storage space. Instead of 
being stored as is, the data is typically subjected to some - 
form of data compression in order to reduce its volume, 
and thereby avoid the high costs associated with storing 
it. "Loss I ess" compression methods such as Lempel-Ziv ' 
Welch (LZWj do not perform ' particularly well on 
scanned pixel maps; While "lossy" methods such as - 
JPEG work fairly "well on continuous-tone pixel maps, 
they do not work particularly well on the parts of the page 
that contain text. To optimize image data compression, 
techniques', whichcan recognize the type of data being 
compressed, are heeded. 

[0003] Known compression techniques are described 
in US-A-5778092, US-A-5251271 , US-A-5060980, US- ' 
A-5784T75, US-A-530331 3'and US-A-5432870. " 
[0004] In one embodiment* the present invention dis- 
closes a method of segmenting a' pixel map represen- 
tation of a document which includes the steps of : acquir-' 
ing a block of the digital image data, wherein r the digital 
image data is composed of light intensity signals in dis- 
crete locations; designating a classification for the block 
and providing an indication about a context of the block; 
segmenting the light intensity signals in the block into 
an upper subset and a lower subset based upon thedes-' 
ignated classification; generating a selector set 'which 
tracks the light intensity segmentation; and separately 
compressing the digital image data contained in the up- 
per and lower subsets? 

[0005] In another embodiment, the : present invention 
discldses a methdd of classifying a block of digital image 
datairitd one of a plurality of image : data types, wherein 
the block of data-is composed of light intensity signals " 
in discrete locations; which includes; dividing the block 
into a bright region ahd'a dark region; dividing a low" pass 
filtered version of the block into a bright' region and a 
dark region; calculating average light intensity values for 
each of the bright region, the" dark region, the filtered 
bright region and the filtered dark region; and comparing 
a differejice_betwe^njhe bright region and the'6atkre- 
gion average light intensity values to a filtered difference 
between the' bright region and the dark region average 
filtered light intensity values; if the average light intensity, 
difference and the average filtered light intensity differ- 
ence are approximately equal finding a range of values 
in which the difference value falls, and classifying the 
block based upon the value range; and if theaverage 
light intensity difference and the average filtered light in- 
tensity difference are not approximately equal finding a 
range of values in which the filtered difference value falls 
and classifying the block based' upon the filtered value 
range. 
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[0006] Some examples of methods according to the 
present invention will now be described with reference 
to the accompanying drawings, in which:- ' 

Figure 1 illustrates a composite image and includes 
ah example of how such an image may be decom- 
posed into three MRC image planes- ah upper 
plane, a lower plane, and a selector plane; 
Figure 2 contains a detailed view of a pixel map and 
the manner in which pixels are "grouped to form 
block's;' : 
Figure 3 contains a flow chart which illustrates gen- 
erally, the steps performed to practice the invention;' ' 
Figure 4 contains a detailed illustration of the man-, 
ner in which blocks may be classified according to" 
the present invention; 

Figure 5 contaihs'a detailed illustration of the man- 
ner in which blocks may be segmented based upon 
their classification according to the present inven- 
tion; ' ' 
Figure 6 contains the details of brie embodiment of 
the manner in which block variation can be meas- 
ured as required by the embodiment'of the invention 
shown in Figure 4; 

Figure 7 contains the details of an embodiment of 
the invention describing classification of blocks 
based upon the block variation measurement pro- 
vided in Figure 6; 1 

Figure 8 contains the details of an embodiment of 
the invention for which context may be updated 
based upon the block classification provided in Fig- 
ure 7; and, 

Figure 9 contains the details of another embodi- ' 
"merit of the invention for updating context based up- 
on block classification as provided in Figure 7. 

[0007]" the present invention is directed to a method 
and apparatus for separately processing the various 
types' of data contained in a composite image. While the 
invenliori' will described in a Mixed Raster Content 
"(MRC) technique, it may be adapted for use with other 
methods and apparatus' and is not therefore, limited'to * 
a MRC format. The'technique described herein is suit- 
able for use in various devices required for storing or 
transmitting documents such as facsimile devices, im- 
age storage devices andthe.like, and.processing of both 
color arid grayscale black arid' white imagesare possi- 
ble/"' ' ' ... 

[0008]' A pixel map is one in which each discrete lo : 
cation on the page contains a picture element or "pixel", 
that erhits a light .signal with a value that indicates'. the 
color or; in the case of gray scale documents, how light 
or dark the image is at that location. As those skilled in 
the art will appreciate, most pixel maps have values that 
are taken from a set of discrete, non-negative integers. 
[0009] For example, in a pixel map for a color docu- 
ment, individual separations are often represented as 
digital values, often in the range 0 to 255, where 0 rep- 
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resents no colorant (i.e. when CMYK separations are 
used), or the lowest value in the range when luminance- 
chrominance separations are used, and 255 represents 
the maximum amount of colorant or the highest value in 
the range. In a gray-scale pixel, map .this typically trans- 
lates to pixel values which range from 0, for. black, to 
255, tor the .whitest tone possible. The. pixel jmaps of 
concern in the currently preferred embodiment of the 
present invention are representations of. "scan qe$" im- 
ages. That is, images which are created by digitizing 
light reflected off of. physical. media using a digital scan- 
ner. The term bitmap js used.to mean a binary pixel rjnap 
in which pixels can .take" one of two values, 1 or 0, 
[0010] Turning now to the drawings for a more de- 
tailed description of the MRC format, pixel map 10 rep- 
resenting a color or gray-scale document is preferably 
decomposed into a three plane page format as indicated 
in Figure 1. Pixels "oh pixel map 10 are preferably 
grouped in blocks 18 (best illustrated in Figure 2), to al- 
low for better image processing efficiency. The docu- 
ment format js typically comprised of an upper plane 1 2, 
a lower plane 1.4, and a selector plane 16. Upper plane 
12 and lower plane 14 contain pixels that describe the 
original image data, wherein pixels in each block. 18 
have been separated based upon pre-defined criteria. 
For example, pixels that have values above a certain 
threshold may be placed on one plane, while those with 
values that are equal to or below the threshold are 
placed on the other plane. Selector plane 1 6 keeps track 
of every pixel in original pixel map 1 0 and maps all pixels 
to an exact spot on either upper plane 12 or lower plane 

14. ( . . ^ ; ' 

[0011] The upper and lower planes are stored at the 
same bit depth and number of colors asthe original pixel 
map 10, but possibly at reduced resolution. Selector 
plane 16 is created and stored as a bitmap. It is imppr-,, 
tant to recognize that while the terms "upper* and "lpw-. : 
er" are used to describe the planes on which data. re--, 
sides, it is not intended to limit the invention to any,par^ 
ticular arrangement or configuration. , :p 

[0012]' After* processing, all three planes are com- 
pressed using a method suitable for the. type of data.re- 
siding thereon. For example, upper plane 1 2 and lower 
plane 14 may be compressed* and stored usjng a lossy' 
compression technique such as JPEG./white selector 
plane 16 is compressed and stored, using a lossless., 
compression format such as gzip or CCITT-G4. it would 
be apparent to one of skill in the art to compress and 
store the planes using other formats that are suitable for 
the intended, use of the output document. For example, 
in the Color Facsimile arena," group 4 (MMR) would pref- 
erably be used for sejector plane 1 6, since the particular 
compression format used must be one of the approved 
formats (MMR, MR, MH, JP£G, JBIG, etc.) for facsimile 
data transmission. 

[0013] In the present invention digital image data is 
preferably processed using a MRC technique such as 
described above. Pixel map 10 represents a scanned 
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image composed of light intensity signals dispersed 
throughout the separation at discrete locations. Again, 
a light signal is emitted from each of these discrete lo- 
cations,, referred to. as "picture elements," "pixels" or 
"pels," at an intensity level which indicates the magni- 
tude of the light being reflected from the original image 
at the corresponding location in that separation. 
[0014] In'typical MRC fashion, pixel map 1P must be . 
partitioned into two planes 12and 14. Figure 3 contains 
a schematic diagram, which outlines the overall process 
used to segment pixel map 10 into an upper plane 12 
and a lower plane 14 according to the present invention. - 
Block 1B is acquired as indicated in step 210; and is clas- 
sified as indicated ,in step 220. in the. preferred embod- 
iment of the inv«ntion,.,block tawiil initially be .classified 
as either UNIFORM, SMOOTH, ' WE AK_ED ( GE or 
EDGE, and its context - either TEXT or PICTURE - yvill 
be provided. The block will then be reclassified as either 
SMQOTH or EDGE, depending upon the initial classifi- 
cation and the context. Next, pixels in block 18 are. seg- 
mented - placed on either upper plane 1 Z or lower plane* 
14 according.to criteria. that is most appropriate forjhe, 
manner in whic.h the block has been classified as indi- 
cated in step 230. This process is repeated for each , 
block 18 in. original pixel map 10 until the entire pixel 
map 10Jias been processed. Upper plane J12, lower 
plane 1 4 and selector plane 1 6 are the ; n separately com- 
pressed, using a technique that is most suitable for the 
type of data contained on each, as indicated in step 240. 
[0015] Turing now to Figure 4, generally speaking, 
classification of blocks 1 8 into one of the. four categories 
in step 220 as described above is prejerably ; completed 
in three steps. First.the variation of^pixel values within . 
■ the block is determined as indicated \n step 31.0, Block, 
variation is best determined .by ( ..using statistical meas- 
ures, which will be described in detaiF telpwwith refer- 
ence to Fig ure\6. Blocks with large : v^aat^opsthroughout 
are most likely to actuallyjie along,edge.s of the image, 
while those containing little. variations probablylie in uni? 
form or at least smooth areas. Measuring the variations- 
within the block allows an inftial.classificatidn. ! tp.be i as-. 
signed .to it as Indicated, in step" 320.. Next, imagedata 
within each block 1 8 is reviewed in.ddtailtoallow context 
information (i.e. ,whether .the, region is ip the text or pic-, 
ture region of .the image) to.be updated^and any neces- 
sary block re-classifications to be performed as shown 
in step 330. The UNIFORM blocks are reclassified as 
SMOOTH, and the WEAK EDGE blocks are upgraded 
to EDGE in a TEXT context or reclassified as SMOOTH 
a PICTURE context. A smoothed version 20 of the 
image is also provided by applying a low pass filter to 
the pixel, map 10. Smoothed image 20 is used in con- 
junction with original image data to offer additional in- 
formation during classification, and also provides un- 
screened data for halftone regions. .. 
[0016] Figure 5 contains details of the manner in 
which block 1 8 is segmented into two planes, as provid- 
ed in step 230 of Figure 3. The measurement begins by 
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first determining at step 410 whether the block being 
processed has initially been classified as an EDGE in 
step 220. If so, the values v p of each pixel in the block 
are first compared to a brightness threshold value L,, 
wherein pixels that have values equal to or above tg are 
viewed as "bright" pixels, while those with values below 
t s are "dark" pixels. Segmenting EDGE blocks simply 
includes placing dark pixels oh upper plane 12 as ihdi- ; 
cated in step 440, and placing bright pixels on lower 
plane 14 as indicated in step 450: If it is determined at 
step 410 that" block 18 is not an EDGE; all pixels in* the 
block are processed together, rather than on a pixel by 
pixel basis. Segmenting of SMOOTH (non-EDGE) pix- 
els occurs as follows: if block 18 is in the midst of a short 1 
run of blocks that have been classified'as SMOOTH, 
and further, all blocks in this short run are dark (v p <t) - 
all data in the block is placed on upper plane 12. If the 
entire block 1 8 is substantially smooth (i.e. in a long run) 
or is bright (in a short run of bright pixels), all data in 
block 18 is placed on lower plane 14. 
[001 7] Turning now to Figure 6, the details of one em- 
bodiment of the invention wherein initial block classifi- 
cation via' block variation measurement may be* accom- 
plished as required by step 310 (Figure 4) f are now de- 
scribed. A threshold, t s , which allows the block to be di- 
vided into two portions is first calculated as indicated in 
step 510. In the preferred embodiment of the invention, 
this threshold is obtained by performing a histogram 
analysis on the data in the block, but many standard 
methods can be used to perform this analysis. For ex- 
ample, the value that maximizes between distances of 
the criteria being used for separation or provides for 
maximum separation between the two portions of the 
block can be selected. Those skilled in the art will rec-' 
ognize that other methods of choosing the best thresh- 
old are available and the invention is not limited to this 
embodiment: Block* 18 is then thresholded into these 
two parts by comparing the light intensity value of each 
pixerto the selected threshold t s , as indicated in step 
520. As before, if 'the 'pixel value v p is less than the 
threshold, the pixel is referred to as dark. If v p is greater 
than'or ecjual to' t s , the' pixel is bright. ' ~ 
[0018] \ As stated earlier, a smooth version 20 of the 
image is obtaihed'by applying a'low pass filter to~the 
original image data. Average values for bright and dark 
p ixels' are then obtained for both the original and 
smoothed sets of image data. Looking first at the bright 
pixels, one' value calculated will be v BP | XEL , the average 
value for all of the bright pixels in original pixel map 10 
( V P 3 * s ) which are located in the area covered by block 
18 a"6 indicated in step 540. Another value ^v B smooth> 
the average value for all of the bright pixels in smoothed 
version 20 of the image which are Ideated in the area 
covered by block 18 will also be obtained as shown in 
step 560. Dark values are calculated similarly. That is, 
v dpixel» tne average value for all of the dark pixels in 
original pixel map 10 (v p < t s ) which are located in the 
area covered by block 18 will be obtained as shown in 



step 550, and v DSMOOTH , the average value for all of the 
dark pixels in the smoothed version 20 of the image 
which are located in "the area covered by block 18 will 
be obtained as in step 570. Once these average values 
5 are obtained, the distances d and d s between brighter 
and darker averages for pixel map 1 0 and smoothed im- 
age" 20 respectively are calculated as indicated in step 
580: That is d= Vbpixel" v dpixeu and d £ = v BSMOO f H - 
v dsmooth Since d/d s is typically almost equal to 1 for 

' 10 contone irriages, the ratio of d/d s may be used to detect" 
halftones, / ' - ■ ' - ■ ■ ■ 

[0019] Figure 7 contains a detailed illustration of step 
320, of Figure 4, the preferred embodiment of a process 
for initially classifying blocks 18. As' shown, a relative 

is comparison between d and d s is obtained as indicated 
in step 6 T0 in order to determine whether the block con- 
tains contone (d » d s ) or halftone data. Block 18 will in- 
itially, be classified as one of four types: . UNIFORM, 
SMOOTH, WEAK EDGE or EDGE according to the* 

20 magnitude of the distance-d 6r d s . Distance d is used io 
classify contone blocks; 'while distance" d s is used for 
halftones. For contone data d, the value from pixel map " 
10, is compared to value x<j as shown in step 620. 
[0020] If d is very low (i.e. d< x 0 ), all pixel : values in 

2S the block are substantially the same and the block is 
classified as UNIFORM at step 640. If there are some- 
what small differences in pixel values in the block such" 
that x 0 <d<x 1 as shown in step 622, the block is classified 
as SMOOTH, at step 650. If there are fairly large differ- 

^0 ences in pixel values in the block and x.,<d<x 2 at step 
624, the block will be classified as WEAK EDGE. If the 
differences in the block are very large and d 3 x 2 at step 
624, the blodk will be classified as an EDGE at step 670. 
[0021] If d/d s is not approximately equai to 1, (i. is 

35 compared to threshold y 0 at step 630. It should be noted 
" there that two different sets of thresholds are applied for 
halftones and contohes. Thus, on most occasions, 
x 0 1 yp; x^y^, and x 2 1 y2- The process usedto classify 
halftone blocks' is similar to that used for contone data. 

40 Thus/ if d s <y 0 at step 630 the block is classified as UNI- 
FORM at step 640. If y p <d s <y 1 in' step 632, the" block is 
classified as SMOOTH at step 650. If y., <d s <y 2 as ihdi : 
cated in step 634, the block is classified as a WEAK 
EDGiE at step 660. If d^ at step 634 : "the block will be 

45 classified as an edge at step 670 i ' 

[0022]. deferring now to Figures 8 and 9,_the details 
for updating the context of the block will now be provid- 
ed. The context of a block is useful when the average \ 
between the dark and bright areas of the block is rela- ' 

50 lively high. When this Is the case,' the block can classi- 
fied as ah EDGE as long as its context is TEXT. The 
context is initially set equal to PICTURE. It is changed 
to TEXT if one of two rules is satisfied: (1 ) the block be- 
ing processed is in a long run of UNIFORM blocks and 

55 the average of the dark pixel values in the block is great- 
er than a preset brightness threshold; or (2) the block 
has been classified as either UNIFORM, WEAK EDGE, 
or EDGE, one of the top, left or right neighboring blocks 
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has a context which has been set equal to TEXT, and 
the difference between that neighboring block and the 
current block is smaller than a preset propagation 
threshold.. 

[0023] Turning first to Figure 8, determining whether . 
block context. should be changed according to the first*, 
rule requires finding a run of blocks that have been clas- - 
sified as UNIFORM as indicated in step 704. Finding a 
run. of UNIFORM blocks typically involves comparing ., 
the number of consecutive UNIFORM blocks to. ; a _run 
length threshold t^j as indicated in .step. 706. .The run v 
length threshold sets thenumber of consecutive blocks 
that'must be classified as JJN I FORM, for, a run to be .es- 
tablished. As also indicated in step 706, V DP | XELl the av- 
erage value of the dark pixels for consecutive blocks Js 
compared tothe brightness threshold t s . A large number . 
of consecutive UNIFORM blocks with high brightness, 
levels usually indicates that the blocks" .contain large 
background page areas (i.e. large white areas), thereby 
indicating that text is present. Thus, if the number of con- 
secutive UNIFORM, blocks exceeds .t LlJ and V D p tX Eu. >_ . 
t s , the context for the block is chahged to TEXT as indi- - 
cated in step 708. . - ■ 

[0024]" If either the number of, identified consecutive 
blocks is too small to establish a run or the blocks are , 
dark (V DPIXEL £ t s j, the context will remain set equal to 
PICTURE. Whether additional runs are present in the . 
block will be determined as indicated in step 7 10,. and if 
so the process will be repeated as indicated in the illus r 
tration. , , 

[0025] Turning now to Figure 9, changing the context' , 
of a block "to TEXT under the second ru'e first, requires-, 
providing a propagation threshold tp. The propagation.,^ 
threshold defines the level of brightness that will indicate ^ 
that the block covers blank page^areas^Underthe sec-^,' 35 
ond rule, the context will be changed from picture to text 
at step 808 if the block is not SMOOTH (i.e. is UNU ' 
FROM, and EDGE or a WEAK EDGE) as shown in step ^ 
802! either its top, left or right neighbor has a text cprjit%xt f . v 
as indicated in step 804 and v BDlF , the average"diff^rV. o 40 
ence between bright pixels in the block and bright pixels. . . 
in the neighbor text, context block is less tha/i "t p as^ 
shown jn step 806. Neighbor blocks are checked -be-" _\ 
cause presumably blocks that contain. text will Ipe l.ocat-7 
ed next to other blocks that contain text'.. However, the., 
brightness value of the block is compared to that of its . 
neighbor to assure that this is the case. In other words, 
even if the block has a neighboring block' with a text con-" 
text, a large difference between the average brightness 
of block* and its neighbor means that the block contain, 
does not contain the large blank page'areas that indicate 
the presence of text. ( * „. 

[0026] Again, the'present invention is directed to seg- 
menting the data by first identifying blocks that contain 
the edges of the image and then separating the blocks 
such that those which contain the* smooth data and 
bright sides of the edges are placed on the lower plane 
and the dark sides of the edges are placed on the upper 



10 



15 



20 



25 



30 



45 



50 



55 



plane. Once each of the respective planes is generated, 
ordinary. MRC processing continues. That is, each plane 
is compressed using an appropriate compression tech- 
nique. In the currently preferred embodiment, upper 
plane 12 and lower plane 1.4 are compressed using 
JPEG while the selector plane 16 is compressed using 
a symbol based pattern matching.technique such as 
CCITT Group IV or a method of classifying scanned 
.symbols into, .equivalence classes such as that de^ 
scribed in US-A 5,778,095. to Davies issued July 7, 
1998, the contents of which ari hereby incorporated by 
reference!.' The planes are then jojne^, together and 
transmitted. to an.output device, such as. a facsimile ma- 
chine or storage, device. . , . , , . . 



Claims , 

1. A method of segmenting digital image data for 
mixed raster content processing, comprising: 

a) acquiring a p\ock pf the digital image data, 
wherein the digital image data is composed of 
light intensity signals in discrete locations; . 

b) designating a classification for said block 
and providing an indication, about a context of 
said block; 

,c) segmenting said light intensity signals in said 
block into an upper subset and a. lower subset 
abased upon said designated classification;, 
d) generating, a selector set which tracks said 
light intensity! segmentation; and 
e j separately compressing the digital image.da- 
ta, contained in said upper arid. lower subsets. 

2. A method, of segmenting dig)taMrriage data as' 
„ claimed, in claim f, .wherein said classification indi- 
., cates that said block .contains Substantially smooth, 
! data arid/or substantially edge dajTa. 

3. A" method of s^jgmebtjng/^igital image data , as 
claimed in.clajm 1 or claim 2, wherein ^said classifi- 
cation data designating^step!furtber9pmprises: ... . 

a) measuring.an.amoun't ol light intensity signal 
variation throughout said block; 

b) assigning a classification to saici block based 
" upon said measured light intensity signal vari- 
ation; and . . 

c) updating said context indication for said 
block, "and designating classification for said 
block based upon said updated context. 

4. A _ method of segmenting digital image -data as 
claimed in any of the preceding claims, further com- 
prising: 

a) dividing a low pass filtered version of said 
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block into a bright region and a dark region; 

b) calculating average filtered light intensity val- 
ues for said bright region and for said dark re- 
gion; and 

c) obtaining a difference in average filtered light 
intensity values between said bright region and 
said dark region. 

A method of segmenting a block of digital image da- 
ta into an upper and lower subset, wherein the block 
of data is composed of light intensity signals in dis- 
crete locations, comprising: 

a) determining whether the block is located on 
an edge in the digital image; 

b) if the block is oh an edge, comparing a mag- 
nitude of each light intensity signal in the block 
to a brightness threshold and placing said sig- 
nal in the upper subset if said light intensity 
magnitude exceeds said brightness threshold 
or in the lower subset if said light intensity mag- 
nitude is less than said brightness threshold; 
and .. . 

c) if the block is not located on an edge, placing 
the block in the upper subset if the block is in a 
group of blocks ;that have light intensity values 
which are indicative ot smooth and dark image 
data, and otherwise placing the block in the low- 
er subset. 

A method of classifying a block of digitaUmage data 
into one of a plurality of image data types, wherein 
the block of data is composed of light intensity sig- 
nals in discrete locations, comprising: 

a) dividing the block into a bright region and a 
dark region; 

b) dividing a low pass filtered version of said 
block into a bright region and a dark region; 

c) calculating average light intensity values for 
each of said bright region, said dark region, 
said filtered bright region and said filtered dark 
region; and 

d) comparing a difference between said bright 
region and said dark region average light inten- 
sity values jo a filtered difference between said 
bright region and said dark region average fil- 
tered light intensity values; 

e) if said average light intensity difference and 
said average filtered light intensity difference 
are approximately equal finding a range of val- 
ues in which said difference value falls, and 
classifying said block based upon said value 
range; and 

f) if said average light intensity difference and 
said average filtered light intensity difference 
are not approximately equal finding a range of 
values in which said filtered difference value 



7. 



falls and classifying said block based upon said 
filtered value range. 

A method according to any of claims 1 to 4, wherein 
blocks are classified by a method according to claim 
5 or claim 6. 
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